Papers with MMLU-pro benchmark
DFPE: A Diverse Fingerprint Ensemble for Enhancing LLM Performance (2026.findings-eacl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) exhibit inconsistent performance across diverse domains. |
| Approach: | They propose a method that systematically constructs subject-adaptive ensembles by balancing model diversity and competence. |
| Outcome: | The proposed method achieves 17.1% gain over the best single model, reaching 71.4% accuracy on the MMLU-pro benchmark. |
COM-BOM: Bayesian Exemplar Search for Efficiently Exploring the Accuracy-Calibration Pareto Frontier (2025.emnlp-main)
Copied to clipboard
| Challenge: | Prior exemplar selection methods focus on maximizing predictive accuracy, neglecting model calibration. |
| Approach: | They propose to use a Bayesian optimization algorithm to optimize for predictive accuracy and calibration. |
| Outcome: | The proposed algorithm beats or matches baselines on multiple tasks from un-saturated MMLU-pro benchmarks while requiring minimal number of API calls. |